Learning with bounded synapses generates synaptic democracy and balanced neurons

Learning in a neuronal network is often thought of as a linear superposition of synaptic modifications induced by individual stimuli. However, since biological synapses are naturally bounded, a linear superposition would cause fast forgetting of previously acquired memory. Here we show that this forgetting can be avoided by additional simple constraints. We consider Hebbian plasticity of excitatory synapses which modifies a synapse only if the postsynaptic response does not match the desired output. With this learning rule the original memory capacity with unbounded weights is regained, provided there is (1) some global inhibition, (2) a small learning rate, and (3) a small neuronal threshold. We prove, in the form of a generalized perceptron convergence theorem, that under these constraints a neuron learns to classify any linearly separable set of patterns. The maximal storage capacity is also reestablished if the synapses are distributed over a spatially extended dendritic tree, provided that distal synapses are allowed to attain stronger weights. After successful learning, excitation will roughly balance inhibition. Moreover, learning a large number of patterns urges the synapses to acquire similar strengths when measured in the soma. The fact that synapses saturate has the additional benefit that non-separable patterns, e.g. similar patterns with contradicting outputs, eventually generate a subthreshold response, and therefore silence neurons which can not provide any information.